What

DDTest now chooses the planned parallel runner count by evaluating a number of possible test splits instead of linearly interpolating from saved/skippable time. Now it is possible to plan the test execution more precise because we have statistics for the durations.

The distribution algorithm plans multiple possible splits between minParallelism and maxParallelism and evaluates them based on (in order of priority):

• expected duration of the slowest worker
• the most even split
• the lowest number of test runners needed

This PR also adds human-readable plan and run reports, printed to stderr by default, so users can understand and debug what DDTest decided. Reports summarize run identity, Datadog feature settings, backend data, planning results, split quality, and worker execution outcome. Reports can be disabled with:

DD_TEST_OPTIMIZATION_RUNNER_REPORT_ENABLED=false

Expected planning report shape:

+++ DDTest: plan report

Run
  Service: checkout-api
  Repository: https://github.com/acme/checkout.git
  Commit: 9f3a1c7d2b4e
  Branch: feature/split-report
  Platform: ruby / rspec
  OS tags: os.platform=linux, os.architecture=amd64, os.version=6.8.0
  Runtime tags: runtime.name=ruby, runtime.version=3.3.4

Datadog
  Test Impact Analysis: enabled
    Test skipping: enabled
    Test impact collection: disabled
  Known tests: enabled
  Impacted tests: disabled
  Early flake detection: enabled
  Auto test retries: enabled
  Flaky test management: enabled

Backend data
  Known tests: 4 modules, 1,284 suites, 18,921 tests
  Skippable tests for this run: 312
  Managed flaky tests: 26 total, 8 quarantined, 3 disabled, 5 attempt-to-fix

Planning
  Test files discovered: 642
  Fully skipped files: 118
  Test files to run: 524
  Duration source: 431 known, 93 default
  Estimated time saved: 38.40%

Split
  Runners: 6
  Expected wall time: 4m12s
  Imbalance: 11s
  Total estimated runtime: 23m46s

Expected run report shape:

+++ DDTest: run report

Run
  Service: checkout-api
  Repository: https://github.com/acme/checkout.git
  Commit: 9f3a1c7d2b4e
  Branch: feature/split-report
  Platform: ruby / rspec
  OS tags: os.platform=linux, os.architecture=amd64, os.version=6.8.0
  Runtime tags: runtime.name=ruby, runtime.version=3.3.4

Execution
  Mode: CI node
  CI node: 2
  Local workers: 2
  Test files run: 87
  Duration: 3m58s
  Result: passed

Why

The previous runner-count calculation could over-provision workers for suites where additional runners did not reduce the slowest runner's estimated wall time. Scoring candidate splits by slowest-runner time, then imbalance, avoids launching workers that would sit idle.

The new reports make the planning decision easier to audit in CI logs. When a split looks surprising, users can see the Datadog settings, available backend data, duration-source coverage, estimated saved time, selected runner count, expected wall time, and imbalance without digging through artifacts or debug logs.

E2E testing

1. Run ddtest plan --platform ruby --framework rspec --min-parallelism 2 --max-parallelism 4 on a project with weighted runnable test files.
2. QA check: confirm stderr contains +++ DDTest: plan report.
3. QA check: confirm the planning report includes run info, Datadog feature settings, backend data, planning counts, duration-source counts, estimated saved time, and split metrics.
4. Inspect .testoptimization/runner/parallel-runners.txt and .testoptimization/runner/tests-split/runner-* to confirm the chosen runner count matches the fastest and most even estimated split.
5. Try different scenarios with different durations for test suites: does the rule hold that we select the split with the lowest wall time and try to make it as even as possible?
6. Run ddtest run --platform ruby --framework rspec using the generated plan artifacts.
7. QA check: confirm stderr contains +++ DDTest: run report.
8. QA check: confirm the run report includes run info, execution mode, CI node/local worker details when applicable, files run, duration, and pass/fail result.
9. Run plan and run again with DD_TEST_OPTIMIZATION_RUNNER_REPORT_ENABLED=false.
10. QA check: confirm neither report is printed while normal artifacts and test execution still work.

Validation performed: go test ./internal/runner, go test ./internal/runner -run '^$' -bench BenchmarkCalculateParallelRunners20000TestFiles -benchtime=1x, make test, and make lint.